Drawing Inference

Lucy D’Agostino McGowan

Is there a difference in frequency score between circle and square beads?

What if I want to know the relationship bead shape and the frequency score?


How can we quantify how much we’d expect the difference in means to differ from one random sample to another?

Is there a difference in frequency score between circle and square beads?

  • We need a measure of uncertainty
  • How about the standard error of the difference in means?
  • The standard error is how much we expect the mean difference to vary from one random sample to another.

Difference in means

\[ \Large \bar{y}_{circle} - \bar{y}_{square} \]

Difference in means

\[ \Large y = \beta_0 + \beta_1(group == square) + \varepsilon \]

Difference in means

\[ \Large \hat{y} = \hat\beta_0 + \hat\beta_1(group == square) \]

lm(frequency_score ~ group, data = sample)

Call:
lm(formula = frequency_score ~ group, data = sample)

Coefficients:
(Intercept)  groupsquare  
      78.50        -8.85

Bead data

How can we quantify how much we’d expect the slope to differ from one random sample to another?

mod <- lm(frequency_score ~ group, data = sample)
summary(mod)

Call:
lm(formula = frequency_score ~ group, data = sample)

Residuals:
    Min      1Q  Median      3Q     Max 
-61.650 -15.500  -3.075  21.350  57.350 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   78.500      7.669  10.235 1.78e-12 ***
groupsquare   -8.850     10.846  -0.816     0.42    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 34.3 on 38 degrees of freedom
Multiple R-squared:  0.01722,   Adjusted R-squared:  -0.008644 
F-statistic: 0.6658 on 1 and 38 DF,  p-value: 0.4196

Bead data

We need a test statistic that incorporates \(\hat{\beta}_1\) and the standard error

mod <- lm(frequency_score ~ group, data = sample)
summary(mod)

Call:
lm(formula = frequency_score ~ group, data = sample)

Residuals:
    Min      1Q  Median      3Q     Max 
-61.650 -15.500  -3.075  21.350  57.350 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   78.500      7.669  10.235 1.78e-12 ***
groupsquare   -8.850     10.846  -0.816     0.42    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 34.3 on 38 degrees of freedom
Multiple R-squared:  0.01722,   Adjusted R-squared:  -0.008644 
F-statistic: 0.6658 on 1 and 38 DF,  p-value: 0.4196
  • \(t = \frac{\hat{\beta}_1}{SE_{\hat{\beta}_1}}\)

Bead data

How do we interpret this?

mod <- lm(frequency_score ~ group, data = sample)
summary(mod)

Call:
lm(formula = frequency_score ~ group, data = sample)

Residuals:
    Min      1Q  Median      3Q     Max 
-61.650 -15.500  -3.075  21.350  57.350 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   78.500      7.669  10.235 1.78e-12 ***
groupsquare   -8.850     10.846  -0.816     0.42    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 34.3 on 38 degrees of freedom
Multiple R-squared:  0.01722,   Adjusted R-squared:  -0.008644 
F-statistic: 0.6658 on 1 and 38 DF,  p-value: 0.4196
  • “\(\hat{\beta}_1\) is -0.82 standard errors below a slope of zero”

Bead data

How do we know what values of this statistic are worth paying attention to?

mod <- lm(frequency_score ~ group, data = sample)
summary(mod)

Call:
lm(formula = frequency_score ~ group, data = sample)

Residuals:
    Min      1Q  Median      3Q     Max 
-61.650 -15.500  -3.075  21.350  57.350 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   78.500      7.669  10.235 1.78e-12 ***
groupsquare   -8.850     10.846  -0.816     0.42    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 34.3 on 38 degrees of freedom
Multiple R-squared:  0.01722,   Adjusted R-squared:  -0.008644 
F-statistic: 0.6658 on 1 and 38 DF,  p-value: 0.4196
  • confidence intervals, p-values
  • Hypothesis testing: \(H_0: \beta_1 = 0\) \(H_A: \beta_1 \neq 0\)

Bead data

How do get a confidence interval for \(\hat{\beta}_1\)? What function can we use in R?


confint(mod)
                2.5 %   97.5 %
(Intercept)  62.97401 94.02599
groupsquare -30.80706 13.10706

How do we interpret this value?

Application Exercise

  1. Calculate the difference in means between your two groups
  2. Create a new project from this template in RStudio Pro:
https://github.com/sta-112-s24/appex-06.git
  1. Fill in your data values for the circle and square beads
  2. Fit the model of frequency_score and group in your data
  3. Calculate a confidence interval for the estimate \(\hat\beta_1\)
  4. Interpret this value
05:00

Hypothesis testing

  • So far, we have estimated the relationship between frequency score and group
  • This could be useful if we wanted to understand, on average, how these variables are related (estimation)
  • This could also be useful if we wanted to a future bead’s frequency score using their group (prediction)
  • What if we just want to know whether there is some relationship bewteen the two? (hypothesis testing)

Hypothesis testing

  • Null hypothesis: There is no relationship between frequency score and group
    • \(H_0: \beta_1 = 0\)
  • Alternative hypothesis: There is a relationship between frequency score and group
    • \(H_A: \beta_1 \neq 0\)

Hypothesis testing

summary(mod)

Call:
lm(formula = frequency_score ~ group, data = sample)

Residuals:
    Min      1Q  Median      3Q     Max 
-61.650 -15.500  -3.075  21.350  57.350 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   78.500      7.669  10.235 1.78e-12 ***
groupsquare   -8.850     10.846  -0.816     0.42    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 34.3 on 38 degrees of freedom
Multiple R-squared:  0.01722,   Adjusted R-squared:  -0.008644 
F-statistic: 0.6658 on 1 and 38 DF,  p-value: 0.4196

Is \(\hat\beta_1\) different from 0?

Hypothesis testing

summary(mod)

Call:
lm(formula = frequency_score ~ group, data = sample)

Residuals:
    Min      1Q  Median      3Q     Max 
-61.650 -15.500  -3.075  21.350  57.350 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   78.500      7.669  10.235 1.78e-12 ***
groupsquare   -8.850     10.846  -0.816     0.42    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 34.3 on 38 degrees of freedom
Multiple R-squared:  0.01722,   Adjusted R-squared:  -0.008644 
F-statistic: 0.6658 on 1 and 38 DF,  p-value: 0.4196

Is \(\beta_1\) different from 0? (notice the lack of the hat!)

p-value

The probability of observing a statistic as extreme or more extreme than the observed test statistic given the null hypothesis is true

p-value

summary(mod)

Call:
lm(formula = frequency_score ~ group, data = sample)

Residuals:
    Min      1Q  Median      3Q     Max 
-61.650 -15.500  -3.075  21.350  57.350 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   78.500      7.669  10.235 1.78e-12 ***
groupsquare   -8.850     10.846  -0.816     0.42    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 34.3 on 38 degrees of freedom
Multiple R-squared:  0.01722,   Adjusted R-squared:  -0.008644 
F-statistic: 0.6658 on 1 and 38 DF,  p-value: 0.4196

What is the p-value? What is the interpretation?

Hypothesis testing

  • Null hypothesis: \(\beta_1 = 0\) (there is no relationship between frequency score and group)
  • Alternative hypothesis: \(\beta_1 \neq 0\) (there is a relationship between frequency score and group)
  • Often we have an \(\alpha\) level cutoff to compare the p-value to, for example 0.05.
  • If p-value < 0.05, we reject the null hypothesis
  • If p-value > 0.05, we fail to reject the null hypothesis
  • Why don’t we ever “accept” the null hypothesis?
  • absense of evidence is not evidence of absense

p-value

summary(mod)

Call:
lm(formula = frequency_score ~ group, data = sample)

Residuals:
    Min      1Q  Median      3Q     Max 
-61.650 -15.500  -3.075  21.350  57.350 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   78.500      7.669  10.235 1.78e-12 ***
groupsquare   -8.850     10.846  -0.816     0.42    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 34.3 on 38 degrees of freedom
Multiple R-squared:  0.01722,   Adjusted R-squared:  -0.008644 
F-statistic: 0.6658 on 1 and 38 DF,  p-value: 0.4196

Do we reject the null hypothesis?

Application Exercise

  1. Open appex-06.qmd
  2. Examine the summary of the model of frequency_score and group with your data
  3. Test the null hypothesis that there is no relationship between frequency score and group
  4. What is the p-value? What is the result of your hypothesis test?
  5. Turn this in on Canvas
02:00

Dr. Lucy D’Agostino McGowan